Optimisations(I WON AGAINST THAT BOT HHAHAHAH)#39
Optimisations(I WON AGAINST THAT BOT HHAHAHAH)#39dumbutdumber wants to merge 14 commits intop-stream:masterfrom
Conversation
Review Summary by QodoDatabase and query optimizations for improved performance and connection stability
WalkthroughsDescription• Replace UUID v4 with UUID v7 for better index locality and write performance • Add database indexes on high-frequency query patterns (user_id, composite keys) • Convert iterative upserts to transactional batches for reduced round-trips • Replace find+create/update patterns with direct upsert operations • Add select clauses to narrow query results and reduce data transfer • Implement PrismaClient singleton pattern to prevent connection explosion • Configure explicit PostgreSQL connection pool with size and timeout settings • Remove redundant pre/post reads around mutations to reduce database chatter Diagramflowchart LR
A["UUID v4 Generation"] -->|"Replace with"| B["UUID v7 Time-Ordered"]
C["Iterative Upserts"] -->|"Convert to"| D["Transactional Batches"]
E["Find + Create/Update"] -->|"Replace with"| F["Direct Upsert"]
G["Full Row Selects"] -->|"Narrow to"| H["Select Specific Fields"]
I["Multiple Connections"] -->|"Implement"| J["PrismaClient Singleton"]
K["Unbounded Pool"] -->|"Configure"| L["Explicit Pool Settings"]
B -->|"Improves"| M["Index Performance"]
D -->|"Reduces"| N["DB Round-trips"]
F -->|"Eliminates"| O["Race Conditions"]
H -->|"Decreases"| P["Data Transfer"]
J -->|"Prevents"| Q["Connection Explosion"]
L -->|"Improves"| R["Load Behavior"]
File Changes1. server/utils/prisma.ts
|
Code Review by Qodo
1. List route missing null-check
|
|
/agentic_review |
|
Persistent review updated to latest commit a624b5a |
|
/agentic_review |
|
Persistent review updated to latest commit 5ee0f9c |
|
/agentic_review |
|
Persistent review updated to latest commit ce98e2f |
|
/agentic_review Hopefully final one :)) |
|
Persistent review updated to latest commit 47de1a9 |
|
/agentic_review Final???? |
|
Persistent review updated to latest commit 118b663 |
|
Most recent commit fixed something pas did :( . In like march or somthing(Ai checked this cause I am not going through all those commits) Pas updated import.ts to use \n instead of null but did not do the same for progress. Now qudo forced me to normalize which is correct so I have done it :)). |
|
/agentic_review FINAL FINAL REVIEW I GIVE UP AFTER THIS |
|
Persistent review updated to latest commit 16f9905 |
|
/agentic_review About to crash out |
|
Persistent review updated to latest commit 8e8fd83 |
|
I know you're fighting with qodo rn but once it's done... do I really need to review this |
Uhh this can be a longer time review, since it does break some features. |
|
//agentic_review Last? |
|
Persistent review updated to latest commit cd6704e |
prisma/migrations/20260301172358_normalize_progress_movie_sentinels/migration.sql
Show resolved
Hide resolved
|
/agentic_review FINAL one |
|
Persistent review updated to latest commit cd6704e |
|
/agentic_review |
|
Persistent review updated to latest commit 8240079 |
|
All bugs show by Qodo are fixed!!!! |
|
To the PR checker person thingamajig: THIS IS A BREAKING PR!!!!!! What will it break?
|
|
Ok a lot to go through, I did a cursory glance at the changes, but I'll need to take a closer look. Overall this cleaner and good work! |
|
okie, the front end changes are not really that big, just how it sends or receives certain data. Also needing to add another check with how the new user and device name is handled. |
Description
This PR fixes/changes a bunch of stuff to the backend that were unoptimized. Look at fixes for DETAILED explanations.
DISCLAIMER: AI was used in the help of making this pr, be it research, identification of bugs or fixes themselves. I can guarantee that its been manually reviewed by me IF it has been generated by AI. Note: This was mainly due to the fact that I found a lot of these fixes before hand and wrote them down and forgot where I kept my text file, after finding it I used an AI agent to do a quick check and do a broader check where I came in and did another double check. I also did use AI to summarize all the changes I made(most of them are simple 1-2 lines changes that just repeat over like 20 files)
Fixes # (issue)
1) PrismaClient singleton fix
Updated Prisma client initialization to reuse a single instance in dev/HMR.
server/utils/prisma.tsIdentified by AI. FIx by me do
2) UUID v7 adoption
Replaced random UUID generation paths with UUIDv7.
^^ The above source is not the one I used(I used a blog but after asking AI cause I could not find the blog again) )
3) Added missing DB indexes
Added hash/composite indexes for high-frequency query patterns.
prisma/schema.prismahttps://www.postgresql.org/docs/16/hash-intro.html
The Type of index was selected by AI, I tried my best to confirm if these were the correct choices however I am not the most confident when it comes to index types so Please be mindful.
4) Connection pool configuration
Added explicit pool size and timeout settings for PostgreSQL adapter usage.
server/utils/prisma.tsThis add a new env called DB_POOL_MAX, this is mainly for the central community DB so the default is quite High for self hosters.
5) Batched loop-based upserts in transactions
Converted iterative upserts to transactional batches where applicable.
Just general optimization made by me :))
6) Converted find+create/update to upsert (progress routes)
Replaced conditional write flows with direct
upsert()logic.server/routes/users/[id]/progress.ts,server/routes/users/[id]/progress/[tmdb_id]/index.tsJust general optimization made by me :))
7) Removed redundant queries
Removed unnecessary pre/post reads around mutations where possible.
Just updates a bunch of queries to use better optimizations also do note that the above source is not the only ones. I used a
bunch for case by case
8) Added
selectclausesApplied narrower
selectprojections for existence checks and payload shaping.Added select for certain calls since we were never using all the data that we got back from the db so its better to just get less info
9) Fixed user deletion cascade coverage
Expanded user delete transaction to include missing related tables.
server/routes/users/[id]/index.tshttps://www.prisma.io/docs/orm/prisma-schema/data-model/relations/referential-actions
U were not deleting certain tables related to some ID when a user deleted account.
10) Enabled relation joins preview support
Enabled
relationJoinsin Prisma schema to support join-based relation load strategy.prisma/schema.prismaNote there is some ups or downs with this for now, I tried simulation an actual db with 50k users and 200 users sending req
every second and my implementation won but in reality it can depend, future update may be needed incase performance drops
11) Idempotency/race-condition hardening
Applied upsert-based patterns to TOCTOU-prone endpoints.
https://www.prisma.io/docs/orm/prisma-client/queries/transactions
This will force all logged in devices when they relog to be registered as NEW devices. Note this was the main AI recc, I confirmed everything and tho it is VERYYYY edge case dependent I still went ahead since edge case handling is good.
Type of change